{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/uptrain-ai/uptrain/blob/main/examples/integrations/llm_providers/together_ai.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1 align=\"center\">\n",
    "  <a href=\"https://uptrain.ai\">\n",
    "    <img width=\"300\" src=\"https://user-images.githubusercontent.com/108270398/214240695-4f958b76-c993-4ddd-8de6-8668f4d0da84.png\" alt=\"uptrain\">\n",
    "  </a>\n",
    "</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using UpTrain with Together AI's Open Source Models\n",
    "\n",
    "[Together AI](https://www.together.ai/) provides fully managed API endpoints for open-source large language models (LLMs), allowing developers for building and running generative AI. The best part is you can use these open-source models directly without worrying about the underlying infrastructure, \n",
    "\n",
    "They offer API endpoints for models like Llama-2, Mistral-7B, CodeLlama, and more. We will use these endpoints to evaluate the performance of these models on a few tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install UpTrain\n",
    "\n",
    "Run the following commands in your terminal to install UpTrain:\n",
    "```bash\n",
    "pip install uptrain\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/shreyanshjain/anaconda3/lib/python3.11/site-packages/lazy_loader/__init__.py:185: RuntimeWarning: subpackages can technically be lazily loaded, but it causes the package to be eagerly loaded even if it is already lazily loaded.So, you probably shouldn't use subpackages with this lazy feature.\n",
      "  warnings.warn(msg, RuntimeWarning)\n"
     ]
    }
   ],
   "source": [
    "from uptrain import EvalLLM, Evals, Settings\n",
    "import json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create your data\n",
    "\n",
    "You can define your data as a simple dictionary with the following keys:\n",
    "\n",
    "- `question`: The question you want to ask\n",
    "- `context`: The context relevant to the question\n",
    "- `response`: The response to the question"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "data = [{\n",
    "    'question': 'Which is the most popular global sport?',\n",
    "    'context': \"The popularity of sports can be measured in various ways, including TV viewership, social media presence, number of participants, and economic impact. Football is undoubtedly the world's most popular sport with major events like the FIFA World Cup and sports personalities like Ronaldo and Messi, drawing a followership of more than 4 billion people. Cricket is particularly popular in countries like India, Pakistan, Australia, and England. The ICC Cricket World Cup and Indian Premier League (IPL) have substantial viewership. The NBA has made basketball popular worldwide, especially in countries like the USA, Canada, China, and the Philippines. Major tennis tournaments like Wimbledon, the US Open, French Open, and Australian Open have large global audiences. Players like Roger Federer, Serena Williams, and Rafael Nadal have boosted the sport's popularity. Field Hockey is very popular in countries like India, Netherlands, and Australia. It has a considerable following in many parts of the world.\",\n",
    "    'response': 'Football is the most popular sport with around 4 billion followers worldwide'\n",
    "}]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get your Together AI API key\n",
    "\n",
    "You can get your Together AI API key [here](https://api.together.xyz/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "TOGETHER_API_KEY = \"*********************\"    # Insert your Together AI API key"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will be using Mixtral-8x7B-Instruct-v0.1 for this example. You can find a full list of available models [here](https://docs.together.ai/docs/inference-models).\n",
    "\n",
    "Remember to add \"together/\" at the beginning of the model name to let UpTrain know that you are using a Together AI model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "settings = Settings(model='together/mistralai/Mixtral-8x7B-Instruct-v0.1', together_api_key=TOGETHER_API_KEY)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create an EvalLLM Evaluator\n",
    "\n",
    "Before we can start using UpTrain, we need to create an EvalLLM Evaluator. You can do this by passing your API key to the `EvalLLM` constructor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "eval_llm = EvalLLM(settings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m2024-02-20 14:52:04.231\u001b[0m | \u001b[33m\u001b[1mWARNING \u001b[0m | \u001b[36muptrain.operators.language.llm\u001b[0m:\u001b[36mfetch_responses\u001b[0m:\u001b[36m236\u001b[0m - \u001b[33m\u001b[1mDetected a running event loop, scheduling requests in a separate thread.\u001b[0m\n",
      "100%|██████████| 1/1 [00:01<00:00,  1.91s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ChatCompletion(id='8585aed94f801d2d-BLR', choices=[Choice(finish_reason='eos', index=0, logprobs=None, message=ChatCompletionMessage(content=' {\\n    \"Reasoning\": \"The extracted context can answer the given query completely. The context provides information about the popularity of various sports and mentions that football is the most popular sport, drawing a followership of more than 4 billion people. Hence, selected choice is A. The extracted context can answer the given query completely.\",\\n    \"Choice\": \"A\"\\n}', role='assistant', function_call=None, tool_calls=None))], created=1708420926, model='mistralai/Mixtral-8x7B-Instruct-v0.1', object='chat.completion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=79, prompt_tokens=799, total_tokens=878), prompt=[])\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "results = eval_llm.evaluate(\n",
    "    data=data,\n",
    "    checks=[Evals.CONTEXT_RELEVANCE]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get your results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "   {\n",
      "      \"question\": \"Which is the most popular global sport?\",\n",
      "      \"context\": \"The popularity of sports can be measured in various ways, including TV viewership, social media presence, number of participants, and economic impact. Football is undoubtedly the world's most popular sport with major events like the FIFA World Cup and sports personalities like Ronaldo and Messi, drawing a followership of more than 4 billion people. Cricket is particularly popular in countries like India, Pakistan, Australia, and England. The ICC Cricket World Cup and Indian Premier League (IPL) have substantial viewership. The NBA has made basketball popular worldwide, especially in countries like the USA, Canada, China, and the Philippines. Major tennis tournaments like Wimbledon, the US Open, French Open, and Australian Open have large global audiences. Players like Roger Federer, Serena Williams, and Rafael Nadal have boosted the sport's popularity. Field Hockey is very popular in countries like India, Netherlands, and Australia. It has a considerable following in many parts of the world.\",\n",
      "      \"response\": \"Football is the most popular sport with around 4 billion followers worldwide\",\n",
      "      \"score_context_relevance\": 1.0,\n",
      "      \"explanation_context_relevance\": \" {\\n    \\\"Reasoning\\\": \\\"The extracted context can answer the given query completely. The context provides information about the popularity of various sports and mentions that football is the most popular sport, drawing a followership of more than 4 billion people. Hence, selected choice is A. The extracted context can answer the given query completely.\\\",\\n    \\\"Choice\\\": \\\"A\\\"\\n}\"\n",
      "   }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "print(json.dumps(results, indent=3))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
